Improved Automatic Discovery of Subgoals for Options in Hierarchical Reinforcement Learning
نویسندگان
چکیده
Options have been shown to be a key step in extending reinforcement learning beyond low-level reactionary systems to higher-level, planning systems. Most of the options research involves hand-crafted options; there has been only very limited work in the automated discovery of options. We extend early work in automated option discovery with a flexible and robust method.
منابع مشابه
Literature Review
Reinforcement learning is an attractive method of machine learning. However, as the state space of a given problem increases, reinforcement learning becomes increasingly inefficient. Hierarchical reinforcement learning is one method of increasing the efficiency of reinforcement learning. It involves breaking the overall goal of a problem into a hierarchy subgoals, and then attempting to achieve...
متن کاملSubgoal Discovery for Hierarchical Reinforcement Learning Using Learned Policies
Reinforcement learning addresses the problem of learning to select actions in order to maximize an agent’s performance in unknown environments. To scale reinforcement learning to complex real-world tasks, agent must be able to discover hierarchical structures within their learning and control systems. This paper presents a method by which a reinforcement learning agent can discover subgoals wit...
متن کاملAutonomous Subgoal Discovery and Hierarchical Abstraction for Reinforcement Learning Using Monte Carlo Method
Autonomous systems are often difficult to program. Reinforcement learning (RL) is an attractive alternative, as it allows the agent to learn behavior on the basis of sparse, delayed reward signals provided only when the agent reaches desired goals. However, standard reinforcement learning methods do not scale well for larger, more complex tasks. One promising approach to scaling up RL is hierar...
متن کاملA novel graphical approach to automatic abstraction in reinforcement learning
Recent researches on automatic skill acquisition in reinforcement learning have focused on subgoal discovery methods. Among them, algorithms based on graph partitioning have achieved higher performance. In this paper, we propose a new automatic skill acquisition framework based on graph partitioning approach. The main steps of this framework are identifying subgoals and discovering useful skill...
متن کاملAutomatic Discovery of Subgoals in Reinforcement Learning using Diverse Density
This paper presents a method by which a reinforcement learning agent can automatically discover certain types of subgoals online. By creating useful new subgoals while learning, the agent is able to accelerate learning on the current task and to transfer its expertise to other, related tasks through the reuse of its ability to attain subgoals. The agent discovers subgoals based on commonalities...
متن کامل